Papers with learning paradigm
Non-Autoregressive Sequence Generation (2022.acl-tutorials)
Copied to clipboard
| Challenge: | Non-autoregressive sequence generation (NAR) models generate output sequences in parallel to speed up generation process. |
| Approach: | This tutorial provides a thorough introduction and review of non-autoregressive sequence generation . it aims to generate the entire or partial output sequences in parallel to speed up the generation process . |
| Outcome: | This tutorial provides a thorough introduction and review of non-autoregressive sequence generation . it aims to reduce the performance gap between state-of-the-art models due to lack of modeling power . |
LLM-driven Instruction Following: Progresses and Concerns (2023.emnlp-tutorial)
Copied to clipboard
| Challenge: | a tutorial on task instruction is aimed at researchers and practitioners interested in NLP generalization . labeled examples are unlikely to be available in large numbers or do not exist . |
| Approach: | This tutorial will examine the progress of natural language processing (NLP) using labeled examples. authors propose that task instructions act as a novel resource for supervision. |
| Outcome: | This tutorial aims to answer questions about instruction-driven NLP . it focuses on the use of task instructions in a low-shot scenario . |
CIKQA: Learning Commonsense Inference with a Unified Knowledge-in-the-loop QA Paradigm (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing commonsense reasoning datasets target different knowledge types, modalities, and formats, but how to help machines acquire and infer over commonsensical knowledge is still unclear. |
| Approach: | They propose a commonsense reasoning benchmark to motivate commonsensing progress from two perspectives: (1) Evaluating whether models can distinguish knowledge quality by predicting if the knowledge is enough to answer the question or not. |
| Outcome: | The proposed model outperforms existing models in evaluating their generalization capabilities across tasks while demonstrating that distinguishing knowledge quality remains challenging for current models. |
Noise Contrastive Estimation-based Matching Framework for Low-Resource Security Attack Pattern Recognition (2024.findings-eacl)
Copied to clipboard
| Challenge: | Techniques, Tactics and Procedures (TTPs) mapping is a difficult task for CTI extraction . conventional learning approaches target the problem in the classical multiclass/label classification setting . |
| Approach: | They propose a neural matching architecture that incorporates a sampling-based learn-to-compare mechanism to facilitate the learning process. |
| Outcome: | The proposed model reduces the complexity of competing over large label space. |
What Makes Instruction Learning Hard? An Investigation and a New Challenge in a Synthetic Environment (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have focused on instruction learning, where a model learns to perform unseen tasks from task descriptions alone. |
| Approach: | They propose to use a controlled synthetic environment to characterize large transformer models as instruction learners. |
| Outcome: | The proposed model can interpret only 65.6% of test instructions and 11%-24% of instructions in out-of-distribution settings. |
Distilling Rule-based Knowledge into Large Language Models (2025.coling-main)
Copied to clipboard
| Challenge: | Recent advances in large language models have broadened their applicability across diverse realworld scenarios. |
| Approach: | They propose to encode rule-based knowledge into large language models by using strong in-context abilities to extract the knowledge from the textual rules and then explicitly encode it into the parameters of LLMs. |
| Outcome: | The proposed learning paradigm is much more efficient than example-based learning in both sample size and generalization ability. |
InstructABSA: Instruction Learning for Aspect Based Sentiment Analysis (2024.naacl-short)
Copied to clipboard
| Challenge: | Experimental results on the Sem Eval 2014, 15, and 16 datasets demonstrate that InstructABSA outperforms the previous state-of-the-art (SOTA) approaches on Term Extraction (ATE), Sentiment Classification(ATSC) and Sentimence Pair Extraction(ASPE) subtasks. |
| Approach: | They introduce positive, negative, and neutral examples to each training sample, and instruction tune the model (Tk-Instruct) for ABSA subtasks. |
| Outcome: | The proposed model outperforms the state-of-the-art (SOTA) on Term Extraction (ATE), Sentiment Classification (ATSC) and Sentimence Pair Extractions (ASPE) subtasks. |
Rethinking Dictionaries and Glyphs for Chinese Language Pre-training (2023.findings-acl)
Copied to clipboard
| Challenge: | Large-scale pre-trained language models (PLMs) such as BERT and GPT have revolutionized various research fields in natural language processing (NLP) |
| Approach: | They propose a new learning paradigm that enhances the semantics understanding ability of Chinese PLMs with dictionary knowledge and structure of Chinese characters. |
| Outcome: | The proposed model improves on both modern Chinese understanding benchmark CLUE and ancient Chinese understanding. |
Embodied Executable Policy Learning with Language-based Scene Summarization (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing Large Language models with text inputs lack the capability to evolve with non-expert interactions with environments. |
| Approach: | They propose a novel learning paradigm that generates robots’ executable actions in the form of text, derived solely from visual observations. |
| Outcome: | The proposed learning paradigm surpasses baselines and can adapt to the target tasks effectively. |
Event Extraction as Machine Reading Comprehension (2020.emnlp-main)
Copied to clipboard
| Challenge: | Event extraction (EE) is a crucial information extraction task that aims to extract event information in texts. |
| Approach: | They propose a new learning paradigm for event extraction by explicitly casting it as a machine reading comprehension problem. |
| Outcome: | The proposed model achieves state-of-the-art performance on the data-scarce scenario, achieving 49.8% in F1 for event argument extraction with only 1% data, compared with 2.2% of the previous method. |
Exploring the Universal Vulnerability of Prompt-based Learning Paradigm (2022.findings-naacl)
Copied to clipboard
| Challenge: | Prompt-based learning inherits the vulnerability from pre-training, where model predictions can be misled by inserting triggers into the text. |
| Approach: | They propose a potential solution to mitigate this vulnerability by injecting triggers into pre-trained language models using only plain text. |
| Outcome: | The proposed learning paradigm inherits the vulnerability from the pre-training stage . it can totally control or severely decrease the performance of prompt-based models . |
ConTinTin: Continual Learning from Task Instructions (2022.acl-long)
Copied to clipboard
| Challenge: | a new learning paradigm is proposed for NLP, which seeks supervision for solving a target task. |
| Approach: | They propose a new learning paradigm that uses textual instructions to learn new tasks . the main goal of machine learning algorithms lies in seeking supervision for solving a target task. |
| Outcome: | The proposed learning paradigm is based on a stream of more than 60 tasks . it makes full use of task instructions to improve forward-transfer and backward-transference . |
Unified Demonstration Retriever for In-Context Learning (2023.acl-long)
Copied to clipboard
Xiaonan Li, Kai Lv, Hang Yan, Tianyang Lin, Wei Zhu, Yuan Ni, Guotong Xie, Xiaoling Wang, Xipeng Qiu
| Challenge: | In-context learning is a new learning paradigm where a language model conditions on a few input-output pairs (demonstrations) and a test input, and directly outputs the prediction. |
| Approach: | They propose a single model to retrieve demonstrations for a wide range of tasks by combining training signals from various tasks into a unified list-wise ranking formulation by language model’s feedback. |
| Outcome: | The proposed model outperforms baselines on 30+ tasks across 13 task families and multiple data domains. |
Transductive Learning for Textual Few-Shot Classification in API-based Embedding Models (2023.emnlp-main)
Copied to clipboard
Pierre Colombo, Victor Pellegrain, Malik Boudiaf, Myriam Tami, Victor Storchan, Ismail Ayed, Pablo Piantanida
| Challenge: | Proprietary and closed APIs are impacting the practical applications of natural language processing. |
| Approach: | They propose a scenario where a pre-trained model is served through a gated API . they propose 'transductive inference' that leverages statistics of unlabelled data . |
| Outcome: | The proposed model performs a few-shot classification task with unlabelled data using a gated API . the proposed model can be used to perform the task with a handful of classes . |
SenticVec: Toward Robust and Human-Centric Neurosymbolic Sentiment Analysis (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to tackle learning challenges such as knowledge forgetting and extensive computing resources are not effective. |
| Approach: | They propose a novel neurosymbolic method for sentiment analysis that places emphasis on human subjectivity within varying domain annotations. |
| Outcome: | The proposed method is lightweight, robust across domains and languages, efficient few-shot training, and rapid convergence. |
Multi-Stage LLM Fine-Tuning with a Continual Learning Setting (2025.findings-naacl)
Copied to clipboard
Changhao Guan, Chao Huang, Hongliang Li, You Li, Ning Cheng, Zihe Liu, Yufeng Chen, Jinan Xu, Jian Liu
| Challenge: | Large language models (LLMs) have made significant progress in knowledge-intensive applications, but they may face a multi-stage continuous learning scenario. |
| Approach: | They propose a multi-stage continuous learning paradigm that includes a preference-based learning bias to identify potential knowledge conflicts and a self-distillation-based data augmentation strategy to expand and enrich the training corpus. |
| Outcome: | The proposed learning paradigm achieves a significant improvement in accuracy after 7 stages of fine-tuning compared to previous methods while preserving general knowledge. |
Chatbot To Help Patients Understand Their Health (2025.findings-emnlp)
Copied to clipboard
Won Seok Jang, Hieu Tran, Manav Shaileshkumar Mistry, Sai Kiran Gandluri, Yifan Zhang, Sharmin Sultana, Sunjae Kwon, Yuan Zhang, Zonghai Yao, Hong Yu
| Challenge: | NoteAid-Chatbot is a conversational AI designed to help patients better understand their health . |
| Approach: | They propose a new learning paradigm that leverages a multi-agent large language model and reinforcement learning framework without relying on costly human-generated training data. |
| Outcome: | The proposed framework surpasses non-expert human training methods. |
It is Simple Sometimes: A Study On Improving Aspect-Based Sentiment Analysis Performance (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing studies have focused on developing ad hoc models of varying complexity for ABSA subtasks. |
| Approach: | They propose an instruction-based model with task descriptions followed by in-context examples on ABSA subtasks. |
| Outcome: | The proposed method outperforms state-of-the-art methods on most domains and achieves competitive results on biomedical domain datasets. |
Finding Support Examples for In-Context Learning (2023.findings-emnlp)
Copied to clipboard
| Challenge: | In-context learning is a new learning paradigm where a language model observes a few examples and directly outputs the test input’s prediction. |
| Approach: | They propose a method to find “support examples” for in-context learning by filtering a training dataset and a progressive filtering process to filter out uninformative examples. |
| Outcome: | The proposed method outperforms baselines and shows that each component contributes critically to the improvements. |
In-Context Learning Creates Task Vectors (2023.findings-emnlp)
Copied to clipboard
| Challenge: | In-context learning (ICL) is a powerful new learning paradigm for Large Language Models (LLMs). |
| Approach: | They propose to use a model with a prompt and a query to learn a mapping based on two examples to produce the output. |
| Outcome: | The proposed model can learn functions from a simple structure based on a training set and a single task vector calculated from the training set. |
MultiInstruct: Improving Multi-Modal Zero-Shot Learning via Instruction Tuning (2023.acl-long)
Copied to clipboard
| Challenge: | Experimental results show zero-shot performance on unseen multimodal tasks . instruction tuning has yet to be explored for vision and multimodal task. |
| Approach: | They propose a multimodal instruction tuning benchmark dataset that consists of 62 diverse multimodal tasks in a unified seq-to-seq format covering 10 broad categories. |
| Outcome: | The proposed model performs well on unseen multimodal tasks and is highly scalable. |
Measuring Pointwise 𝒱-Usable Information In-Context-ly (2023.findings-emnlp)
Copied to clipboard
| Challenge: | In-context learning (ICL) is a new learning paradigm that has gained popularity along with the development of large language models. |
| Approach: | They propose to adapt a recently proposed hardness metric, pointwise V-usable information (PVI), to an in-context version. |
| Outcome: | The proposed hardness metric is compared with the original model and is more efficient because it requires only a few exemplars and does not require fine-tuning. |
Towards Informative Few-Shot Prompt with Maximum Information Gain for In-Context Learning (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language models (LLMs) have the capability to engage In-context Learning (ICL) however, this particular learning paradigm suffers from high instability stemming from factors such as input distribution, order and prompt formats. |
| Approach: | They propose to quantify the information gain obtained in prediction after observing a given example candidate and to sample those with maximum IG. |
| Outcome: | The proposed method can yield an average relative improvement of 14.3% across six classification tasks using three LLMs. |